I These slides/notes represent only part of the course, and were 
accompanied by face-to-face explanations on white-board and 
additional topics / learning materials. 


L1 In preparation of these slides I have also benefited from various 
books and online material. 


I Some of the slides contain animations which may not be visible 
in pdf version. 


L] Corrections, comments, feedback may be sent to 
https://www.linkedin.com/in/naveedrazzaabutt 
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Full Courses 


° Stochastic Processes 

* Statistical DSP & 
Modelling 

e Probabilistic Methods in 


Engineering 

° Wave Propagation & 
Antennas 

* Principles of 
Communications 


* Digital Communications 
° Satellite Communications 
* Circuit Analysis II 


As a teacher 





Labs & Tutorials 


* Time Series Analysis 

° Signal Theory 

° Advanced Control 

* Modern Control 
Systems 

e Digital Design 
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Supervision 


Supervised and 
collaborated in 
various grad and 
postgrad theses. 


As a researcher 
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One of my research projects... 
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Question: how to 
quickly tell whether a 
medicine 1s fake? 





ES544 - Random Processes / Dr. Naveed R. Butt @ GIKI - FES 


15 





Compour:d 
under study 





NMIR vs. NOR 


ーー re 
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Compound 
under study 


RF coil 
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Amplitude 


NOR Signal Modeling & Detection 
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The Course ... 


° Why study random processes? 


* Where can I apply what I learn? = 
e How is the course conducted and assessed? 


Basically, wherever you would like to (or 


would benefit from) including uncertainty 
or variations in your mathematical model. 
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Communications 


THIS IS FLIGHT Y7A 
REQUESTING EMERGENCY 
ASSISTANCE... 
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Bio-Medicine 


Dr. Tinycat says your heart rate is normal 


ICAMHASCHEEZEURGER COM Ss S 





Name: 12-Lead2 HR 62 bpm | * Normal ECG “Unconfirmed™ 

ID: 14:37:18 | Normal sinus rhythm 

Patient ID PR B. 138s ORS 8. 112s 

Incident Qr/ürc H 3985/8. 395s 

P Sex eegene 27° 88° 49° T | | 
| ; | | 

ru | | | 


, 
| | | 
x1.8 .85-150Hz 25mm/sec ipu L IL 
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Finance 








REGULATIONS IT'S NOW 
SAFE TO INVEST YOUR 
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XLF (Financials Select Sector SPOR) AMEX 


o Stoch harts com 


10-Aug-2007 Open 32.19 High 35.19 Low 31.95 Close 33 35 Volume 519.8M Chg +1.29 (*4.02%) « 
M) XLF (Weekly) 33.35 38 
—=MALSO) 26.00 | | 





= MAL 20) 35.27 
da Volurne $19,772,480 


PON DE TMAMIJUIASOTSSE?MALG)JJASOENDS FRA M 
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Chemistry 





NOP dea cortsning TNT sampl» 


020 <109 — 420 MM wm 4X0 en en 4909 430 DO 
carck 


ES544 - Random Processes / Dr. Naveed R. Butt @ GIKI - FES 


2200 *300 


MOR data without TINT camici 


no LES 4200 
samp 


a799 





180 410900 


26 


Or, just use it to prove existence of atoms... 





Einstein’s 1905 papers 


]. On a Heuristic Point of View on the Creation 
and Conversion of Light (Photo-Electric Effect) 





2. On the Electrodynamics of Moving Bodies 
(Theory of Special Relativity) 





3. Investigation on the Theory of the Brownian 
Movement 
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Or, just use it to prove existence of atoms... 


Mystery of Brownian Motion 


With a microscope, 
Robert Brown saw 

small pollen grains : 
move continuously <A 
and randomly for | 

no apparent reason. 


Why? 





Einstein's Hypothesis 


Motion of the pollen grains is due to 
collisions with even smaller unseen atoms. 


Einstein said ignore 
each chaotic jitter, 
instead measure the 


average motion over ` 


an extended time. 
"Ze NE 
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The Course ... 


° Why study random processes? 
* Where can I apply what I learn? 
* How is the course conducted and assessed? Ç | 
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Important Business! 


° 80% attendance is mandatory! 


° Textbooks 
* Lindgren, G., Rootzen, H., Sandsten, M., Stationary Stochastic Processes for 


Scientists and Engineers, CRC Press, 2013 
* Kobayashi, H., Mark, B., & Turin, W., Probability, Random Processes, and 
Statistical Analysis, Cambridge University Press, 2011. 


° Contact 
° naveed.butt@giki.edu.pk 
° office: FES G-6 


Weightage 


E Final B Mid Viva Seminar WE Term Paper 


Learning Plan 


* Lectures 
* Help discover and grasp new concepts 

° Viva (Assignment-based) 
* Help prepare/revise each week's concepts 
* Keep you from lagging behind in course 


e Seminar 
* Prepares for presentations and teaching 


* Term Paper 
* Helps practice independent research/review work 
* Helps appreciate wider aspects of course content 
* Exams (Mid, Final) 


* Help prepare entire course material 
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Questions’? Thoughts?? 
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Frequency (hertz) 
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Q. Can we write processes/signals as 
sums of periodic functions 
(frequencies)? 


This is exactly what Fourier Transform 
does — it tries to write every signal as a 
sum of sinusoids. 


Note: for the time being we will focus 
on deterministic processes. 


Q. Can we write processes as sums of 
periodic functions (frequencies)? 
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Baking a Fourier Cake 


° Given: Signal shape (time-domain) 
° Ingredients: Sinusoids of different frequencies 
° Choose: How much of the each ingredient (sinusoid) to use? 





* In Fourier Transform, we want to look at signals in terms of a fixed set 
of ingredients 
* Ingredients : Sinusoids of different frequencies 





* In Fourier Transform, we want to look at signals in terms of a fixed set 
of ingredients 
* Ingredients : Sinusoids of different frequencies 


Ingredient Amount 
(sinusoid frequency) (scaling) 








Y N + 1 Add all 
x ' CD 
/ I" / à Ui — ba a, 0.5 
k. 0.25 
+ 
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e How is this shown after Fourier transform? 





Scaling 

Ingredient Amount 

+ (sinusoid frequency) (scaling) 
5 Hz 1 Add all 1 

10 Hz 0.5 
+ 15 Hz 0.25 Oe 
0.25 
D 10 15 


Frequency (Hz) 
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* We mostly skip the middle steps 


Fourier Transform 


= 
Ss 


Inverse 
Fourier Transform 





Amplitude 





Time 


Scaling 


10 15 
Frequency (Hz) 


MA on 
VUE 


Some Fourier Transtorms (Visual) 


DC 
| | 
t i Ü N-1 O 
Q N- | 0 


L 1 
t f 
Ü M- 1 Ü 
Ü M- 1 Ü 


As frequencies increase, the FT peaks move outwards 
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AMPLITUDE 





AMPLITUDE 


TIME 


ES 544 - Random Processes / Dr. Naveed R. Butt Q GIKI - FES 





17 








Time Domain 


s(t) 





FT Frequency Domain 
—Í : ede 
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harmonics: 1 


0.5 


0 10 20 30 
Frequency (Hz) 
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Time (s) Frequency (Hz) 


ES 544 - Random Processes / Dr. Naveed R. Butt Q GIKI - FES 


23 








lucasvb.tumblr.com 


ES 544 - Random Processes / Dr. Naveed R. Butt Q GIKI - FES 


24 


Q. Is it always possible to write signals as 
sums of sinusoids? 


Q. Is it always possible to write signals as 
sums of sinusoids? 


° No. There are theoretical signals that do not have a 
Fourier Transform (e.g., e “u(t) with a < 0). 


* However, all physically realizable signals have Fourier 
Transforms. 


Q. Why Use Periodic Functions (frequencies)? 


A large number of physical 
phenomena have underlying 
periodicities (frequencies)... 


arm plitude 


flute 





recorder 
e» 923 
a 20 
215 
3 10 
5 2770 wei 
Ü 
Ü 1000 2000 z000 0 1000 2000 3000 4000 
frequency (Hz) frequency (Hz) 
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opectral Magnitude (dB) 


Amplitude 


30 


20 
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2000 Oc, 


F2 F3 F4F5 
Frequency (Hz) 


Fa F4F5 
Frequency (Hz) 








ime (5) 


Voice Recognition 
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RF Level [ Wim? ] 


100 








Background Frequency [ MHz ] 


Communication systems are based on the frequencies of tunable antennas 
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0 1.000 x 2.000 3.000 4.000 5,000 
'4N NOR frequency (kilohertz) 


| Il |! 











Chemicals may be identified by the unique resonant frequencies of their 
nuclei or molecules 
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Q. Why Sinusoids? 


Why not other types of periodic 
functions? 





Triangle 
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Q. Why Sinusoids? 


* Smooth (analytically simpler, e.g., differentiable, integrable...) 
* Nicely reflect behavior of natural phenomena (to-and-fro motions) 


* Sinusoids are Eigen-functions of linear time-invariant operators 
(which helps simplify computations) 


Q. Why Sinusoids? 










-Ñ D ERËN a ] T fy ü Ki ës Dei ` NU (RS š it Ls aa Ab KAN a a _ 
F e ett Am ën Ni l ZC i AH ph Ww e O SE SEN Lä hr. d ON TE A, d a Ñ t ks we ` L Te e 





ES 544 - Random Processes / Dr. Naveed R. Butt @ GIKI - FES 








electromagnetic wave dil 


r 


35 


In fact, Fourier Transform does not 
use just sinusoids, it uses complex 
sinusoids!!! 


Why? 
e More general than real sinusoids 
e More elegant analytically and in calculations 


And what exactly was a complex sinusoid? 


A Sine and a Cosine Walk Into an 
Imaginary Bar... 


The Complex Sinusoid - e 


Jot 





- 7 
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The Complex Sinusoid - e/**t 


cos D 


cos(wt) + j sin(wt) ロニ < 


sin 8 





ry - w mm e e së L= b m-— m pn 
LI d ell cu m [ mm | ` J CHT] 
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The Complex Sinusoid - eJ€9* = coslwt) + jsin(ot) 






Imag axis 


270° 


Time 





(a) 
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Mathematically Speaking... 


x(t) = — J Xl d” dæ 


AT Joo 
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Mathematically Speaking... 


2 

1.5 
05 

1 
0.4 

0.5 
0 03 

05 
02 

-1 
45 S 
2 0 

100 150 200 280 200 350 400 450 


Time Frequency 





X(w) = J x(t)e I” qt 


OO 
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FT — Pairs and Rules-of-Thumb 


Some Fourier Transtorms (Visual) 


Not changing 
(frequency = 0) 


sin(wt) = > (yet ー e Jet) P4 
constant ó(s) 
AAAAA e m 8 
TT 1/T 


Sin wave 
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DC 
— m | | 
t f 0 N-1 o 
0 N-1 0 


t 1 
t f 
Ü M- 1 Ü 
Ü N- 1 Ü 


Rule1: As frequency increases, the FT peaks move outwards 
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constant ó(s) 
(aj 


+”, 


- - T/T 
T Sin wave 


(b 


Rule 2: Damping causes spread 
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|x(t)| 





Ü 5 10 15 20 29 30 f [Hz] 


Rule 3: Sharp changes (edges) require a lot of frequencies 
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"time" domain frequency domain 
Impulse, or "delta" function 


... and an extremely sharp change 
(impulse) requires ALL the frequencies!! 


Rule 3: Sharp changes (edges) require a lot of frequencies 
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| (d) 1 
8 MAM 中 

—| 
(b) I (e) 1 
e F103 Rule 4: Periodic functions 

, have discrete spectra. 
i Milli i "| 

Ri | 

Q 10 20 3( 
49 


Time (s) Frequency (Hz) 


—" 
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Rule 5: Multiple effects 
can be combined. 








4 | 
~ | 
c 9 
0 
0 0.5 1 1.5 2 
t f 
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Rule 6: Duality 


Hi, De. El:zabeth” 
Yeoh, uh... T. accidentally tK 
the enge transform of UL cat... 


LC, 
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Questions??? Thoughts?? 
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